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EXECUTIVE  SUMMARY 


The  purpose  of  this  study  project  was  to  provide  a better  under- 
standing of  the  factors  influencing  any  decisions  to  use  commercial  "off- 
the-shelf"  avionics  equipment  in  military  aircraft.  The  specific  goals  of 
the  project  were:  to  identify  and  evaluate  the  documents  controlling  the 
performance,  environmental  testing  and  the  reliability  testing  of  commer- 
cial avionics  equipment  and  to  compare  these  procedures  with  conventional 
military  practice  (Section  II  through  Section  V)  and  to  analyze  and  high- 
light those  factors  which  impact  the  decision  of  an  acquisition  program 
manager  who  is  considering  the  use  of  commercial  equipment  in  military 
aircraft. 

Background  information  was  obtained  through  interviews  with  Federal 
Aviation  Agency  (FAA)  officials  and  government  engineers.  Applicable  FAA, 
military  and  industry  regulations,  specifications  and  documents  were 
reviewed.  The  data  from  these  interviews  and  the  document  reviews  were 
analyzed  and  conclusions  drawn. 

The  primary  conclusions  of  the  evaluations  were: 

1.  The  FAA  approval  documents  and  the  supporting  specifications 
are  not  intended  for  nor  are  they  suitable  for  use  as  procurement  speci- 
fications. 

2.  Airframe  manufacturers  and  commercial  airlines  purchase 
avior.ics  equipment,  to  airframe  company  specifications,  the  equipment  man- 
ufacturers specifications  or  industry  association  specifications.  These 
specifications  may  be  suitable  for  military  procurements,  but  a careful 
review  must  be  made  to  ensure  that  the  performance  and  environmental  test- 
ing specified  is  adequate  for  the  intended  military  use. 
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3.  Reliability  testing  is  not  normally  required  by  the  FAA  on 


commercial  avionics  equipment. 

4.  When  an  item  of  equipment  is  used  in  both  commercial  and 
military  aircraft,  the  reported  mean  time  between  failure  (MTBF)  in 
military  aircraft  is  normally  less  than  half  the  MTBF  reported  in  comm- 
ercial service. 
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SECTION  I 


Introduction 

Purpose  of  the  Study  Project 

Program  managers  involved  in  the  acquisition  of  aircraft  periodically 
must  decide  if  an  "off-the-shelf"  item  of  commercial  equipment  is  accept- 
able for  the  program  or  whether  an  item  qualified  to  military  specifications 
and  standards  should  be  procured.  The  purpose  of  this  study  project  is  to 
evaluate  the  commercial  equipment  qualification  and  certification  proce- 
dures, compare  these  procedures  with  conventional  military  practice  and 
to  analyze  and  highlight  those  factors  which  impact  the  decision  of  the 
program  manager. 

Specific  Goals  of  the  Project 

Th^  specific  goals  of  the  project  were: 

1.  To  identify  and  evaluate  the  documents  used  to  control  the 
performance  requirements  for  the  design  of  commercial  "off-the-shelf" 
avionics  equipment. 

2.  To  evaluate  the  adequacy  of  the  environmental  testing 
required  by  the  Federal  Aviation  Agency  (FAA)  on  commercial  avionics 
equipment  to  see  if  equipment  so  tested  can  be  expected  to  perform  satis- 
factorily when  installed  in  military  aircraft. 

3.  To  evaluate  the  reliability  testing  required  by  the  FAA  on 
commercial  avionics  equipment  to  determine  if  the  tests  performed  are 
adequate  to  forecast  the  reliability  of  the  equipment  in  military  air- 
craft use. 


1 


4.  To  compare  the  reliability  of-  an  inertial  navigatior  system 
as  reported  in  commercial  aviation  service  with  that  achieved  with  the 
same  equipment  in  military  service. 

5.  To  provide  specific  recommendations  on  factors  that  program 
managers  should  consider  when  deciding  whether  to  use  commercial  "off- 
the-shelf"  equipment. 

Scope  of  the  Project 

This  study  project  is  directed  toward  the  impact  of  performance  stan- 
dards and  reliability  and  environmental  tests  on  the  selection  of  commer- 
cial "off-the-shelf"  avionics  equipment.  The  study  specifically  considers 
two  typical  items,  an  inertial  navigation  system  and  an  altimeter.  The 
controlling  test  documents  are  evaluated  and  in  the  case  of  the  navigation 
system,  comparative  reliability  performance  in  military  and  commercial 
service  are  compared. 

Limitations  of  the  Report 

The  study  considered  only  commercial  avionic  equipment.  Selection 
criteria  from  other  types  of  commercial  equipment  (e.g.,  generators, 
hydraulic  pumps,  tires)  was  not  considered.  Even  for  the  avionics  equip- 
ment within  the  scope  of  this  report,  there  are  factors  influencing  the 
choice  of  equipment  that  are  not  considered.  Among  the  additional  factors 
that  were  not  evaluated  are  cost,  delivery  schedules,  maintainability, 
equipment  and  component  standardization,  and  configuration  management. 
Organization  of  the  Report 

The  various  types  of  performance  requirement  documents  and  qualifica- 
tion methods  are  discussed  in  Section  II.  Sections  III  and  IV  review  and 
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compare  current  environmental  and  reliability  test  qualification  methods. 
Section  V compares  commercial  and  military  reliability  experience. 

Section  VI  analyses  the  impact  of  these  factors  on  decisions  of  the  pro- 
gram manager.  Section  VII  contains  conclusions  based  on  the  study. 
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SECTION  II 

Commercial  Avionics  Equipment  Qualification  Methods 
A primary  factor  in  the  selection  of  commercially  used  avionics 
equipment  for  military  use  is  the  understanding  of  how  this  equipment  has 
been  accepted  for  use  by  the  FAA  and  what  are  the  limits  on  such  use.  A 
brief  description  of  the  primary  methods  of  certification  will  highlight 
some  of  the  conditions  surrounding  the  certification.  The  four  methods 
commonly  used  are: 

1.  Technical  Standard  Order  Authorizations 

2.  Aircraft  Type  Certificate 

3.  Supplemental  Type  Certificate 

4.  Any  other  means  approved  by  the  Administrator 
Technical  Standard  Order  Authorization  (TSO's) 

The  general  procedures  governing  TSO  approval  are  contained  in  the 
Code  of  Federal  Regulations  Title  14,  (Aeronautics  and  Space)  Part  37 
Technical  Standard  Order  Authorizations  (14  CFR  37).  This  directive  is 
commonly  called  Federal  Aviation  Regulation  37  (FAR  37).  The  crucial 
portion  of  Part  37  from  a program  manager's  viewpoint  is  that  it  contains 
"minimum  performance  and  quality  control  standards  for  specified  materials, 
parts,  or  appliances. .. .used  on  civil  aircraft."  (Part  37.1  (a)(2)). 

The  key  operative  words  are  minimum  performance  standards . These  are 
interpreted  to  mean  those  performance  conditions  that  permit  safe  per- 
formance of  the  aircraft.  This  does  not  mean  that  the  performance  is 
optimized  for  any  particular  aircraft  installation,  just  that  installation 
of  the  equipment  will  not  result  in  unsafe  operations. 
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A manufacturer  desiring  to  receive  TSO  approval  on  an  item  must: 

1.  Conduct  necessary  tests; 

2.  Submit  technical  data  required  by  the  applicable  performance 
standards  of  14  CFR  37; 

3.  Submit  a description  of  his  quality  control  system; 

4.  Maintain  a current  and  complete  technical  file  on  the 

article; 

5.  Mark  the  article  with  specified  information. 

The  applicable  performance  standards  mentioned  are  standards  speci- 
fied in  14  CFR  37  for  a particular  item  of  equipment,  e.g..  rate  of 
climb-TSO -C9c,  life  preservers-TS0-C13c,  fuel  flowmeters-TS0-C44a. 

The  performance  standards  mentioned  in  individual  TSO's  are  in  some 
cases  contained  in  their  entirety  in  14  CFR  37,  but  are  more  commonly  con- 
tained in  documents  referenced  by  14  CFR  37  with  the  additions  or  deletions 
deemed  appropriate  by  the  FAA. 

References  include  a few  military  specifications  and  many  aeronautical 
standards  prepared  by  industry  groups  with  government  participation.  The 
three  most  common  groups  preparing  avionics  equipment  performance  standards 
are  the  Society  of  Automotive  Engineers  (SAE),  Radio  Technical  Commission 
for  Aeronautics  (RTCA),  and  Aeronautical  Radio  Incorporated  (ARINC). 

The  standards  reflect  the  desire  of  the  FAA  to  require  items  that 
will  produce  an  acceptable  level  of  safety  for  all  aircraft  without 
unnecessarily  increasing  the  cost  of  light  private  aircraft.  The  typical 
standard  specifies  materials,  detailed  design  requirements,  environmental 
conditions  and  test  conditions.  While  the  standards  are  adequate  to 
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ensure  safety,  they  do  not  always  reflect  current  design  practices.  For 
example,  the  SAE  has  published  Aeronautical  Standards  that  are  used  as 
TSO  references  in  Part  37,  but  in  some  cases  the  SAE  committee  has  decided 
that  the  FAA  requirements  are  inadequate  as  a design  guide,  and  has  pub- 
lished a Recommended  Aeronautical  Practice  document  with  more  stringent 
criteria. 

Update  of  the  TSO's  is  a continuing  process  complicated  by  the  broad 
range  of  aircraft  types  involved,  numerous  manufacturer  and  user  associa- 
tions with  strong  preferences  and  vested  interests,  and  an  approval  pro- 
cedure requiring  substantial  time  for  comment  and  protest  after  the 
publication  of  a proposed  rule  change. 

Some  of  the  TSO  references  are  very  dated  (e.g.,  Turr,buckles-TS0-C21a 
is  a 26  year  old  document).  Many  of  the  instrument  related  TSO's  are 
based  on  Aeronautical  Standards  dated  in  1959.  On  the  other  hand,  the 
FAA  has  created  TSO's  for  new  types  of  equipment  in  a very  limited  time 
as  evidenced  by  the  rapid  reaction  to  provide  standards  for  the  Ground 
Proximity  Warning  system. 

One  important  fact  is  that  few  items  with  TSO  approval  met  just  the 
minimum  standards  set  forth  in  the  TSO.  Most  instruments  with  TSO 
approval  have  performance  characteristics  that  exceed  the  standards  by  a 
substantial  amount.  For  some  equipment,  major  aircraft  manufacturers 
develop  specifications  that  are  as  detailed  as  government  specifications. 

In  addition  to  requiring  compliance  with  the  company  specification,  the 
aircraft  manufacturer  will  require  the  instrument  manufacturer  to  obtain 
TSO  approval  from  the  FAA..  Obviously  net  all  items  approved  under  the  same 
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TSO  are  equivalent.  There  can  be  substantial  differences  in  performance 
among  instruments  meeting  a given  TSO. 

In  addition,  the  TSO  does  not  control  the  size  or  the  interface 
characteristics.  The  primary  reason  for  this  is  that  the  TSO  has  never 
been  intended  as  a procurement  document. 

The  airline  industry  benefits  from  the  standarization  of  equipment. 
Aeronautical  Radio  Inc.  (ARINC)  is  primarily  funded  by  the  commercial 
airlines.  ARINC  has  published  a numoer  of  standards  detailing  performance 
requirements  for  avionics  equipment.  These  documents  also  specify  elec- 
trical and  mechanical  interface  requirements  and  test  requirements.  If 
the  equipment  is  of  a class  covered  by  a TSO,  the  ARINC  document  incor- 
porates the  TSO  requirements.  ARINC  documents  have  provisions  for  addi- 
tional equipment  capability  where  appropriate.  ARINC  documents  have  been 
used  by  both  the  government  and  industry  as  procurement  specifications. 

A quick  comparison  on  two  altimeters,  one  built  to  the  minimum  require 
ments  of  TSO-C-lOb,  and  the  other  conforming  to  the  requirements  of  Mil-A- 
83212  Altimeter  (AAU-27/A)  specification,  will  illustrate  some  of  the 
differences.  The  basic  accuracy  of  the  two  instruments  (altitude  vs. 
pressure)  is  reasonably  comparable  at  room  temperature  with  the  TSO  in- 
strument being  10  to  15  feet  more  accurate  below  8,000  feet  altitude,  and 
the  AAU-27  more  accurate  by  up  to  90  feet  at  higher  altitudes.  Many  tests 
are  similar  for  both  instruments,  but  the  following  tests  are  called  out 
for  the  AAU-27/A  and  are  not  required  for  the  TSO  instruments. 


1. 


2. 


Maintainability  Test 
Reliability  Test 
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3.  Longevity 

4.  Operational  Stability 

5.  Lighting 

6.  Over-voltage  Protection 

7.  Undervoltage  Protection 

8.  Warmup  Time 

9.  Dielectric  Strength 

10.  Backlash 

11.  Acceleration 

12.  Shock 

13.  Shelf  Life 

14.  Response  (lag) 

In  addition,  the  display  characteristics,  viewing  angle,  dimensions 
and  the  mechanical  and  electrical  interface  are  defined  for  the  AAU-27 
but  not  for  the  TSO  instrument.  These  last  items  reflect  the  fact  the 
AAU-27  specification  is  a procurement  document  while  the  TSO  reference 
is  not  intended  for  that  purpose.  The  differences  in  environmental  tests 
are  covered  in  Section  III. 

When  a major  airframe  manufacturer  (Boeing,  McDonnell  Douglas, 
Lockheed,  etc)  buys  a TSO  altimeter,  the  procurement  specification  exceeds 
the  TSO  minimum  requirements  and  has  tests  parelleling  most  if  not  all  of 
the  military  specification  requirements. 

Aircraft  Type  Certificate 

The  second  method  of  item  certification  is  as  a part  of  the  entire 
aircraft.  During  the  development  of  the  aircraft  and  its  rented  equip- 
ment, the  FAA  monitors  the  design  and  technical  verification  of  the  entire 


aircraft  including  the  avionics  items.  The  primary  goal  is  to  assure  that 
the  item  installed  meets  minimum  performance  standards  under  the  conditions 
that  it  will  be  used  in  the  new  aircraft.  While  the  instrument  does  not 
have  to  be  TSO'd,  the  FAA  must  be  convinced  by  the  data  produced  that  a 
safety  level  at  least  equivalent  to  that  provided  by  a TSO  instrument  will 
result. 

After  the  aircraft  has  satisfactorily  passed  all  applicable  ground 
and  flight  tests,  a Type  Certificate  is  granted  for  the  aircraft.  The 
Type  Certificate  contains  an  approved  parts  list  giving  the  make,  model 
and  dash  number  of  the  approved  equipment.  Items  receiving  approval 
under  the  Type  Certificate  are  approved  only  for  use  on  that  particular 
aircraft  type.  In  this  case,  the  airframe  manufacturer  rather  than  the 
equipment  manufacturer  is  responsible  for  ensuring  that  all  produced 
items  meet  the  performance  standards  that  applied  during  the  certification 
process. 

Supplemental  Type  Certificate 

The  procedure  followed  before  a Supplemental  Type  Certificate  is 
granted  parallels  very  closely  the  procedure  for  the  lype  Certificate. 
Supplemental  Type  Certificates  are  granted  following  a major  aircraft 
modification  by  either  the  original  manufacturer  or  another  company.  Any 
items  changed  are  listed  on  the  Supplemental  Type  Certificate. 

For  both  the  Type  Certificate  or  Supplemental  Type  Certificate, 
alternate  items  may  be  placed  on  the  Authorized  Equipment  List  upon  sub- 
mission of  sufficient  evidence  to  convince  the  FAA  that  satisfactory  safety 
standards  have  been  maintained.  Many  of  these  alternate  items  are  approved 
based  on  similarity  to  the  approved  equipment. 
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Environmental  Tests 


The  environmental  testing  philosophy  for  the  military  specified 
equipment  and  for  commercial  equipment  approved  by  the  FAA  is  quite 
similar.  The  methods  of  application  may,  however,  impact  on  the  suit- 
ability of  a given  item  for  a specific  application.  Military  specified 
tests  are  listed  MIL-STD-810C,  Environmental  Test  Methods.  The  civilian 
equivalent  is  Radio  Technical  Commission  for  Aeronautics  (RTCA)  Document 
No.  160,  Environmental  Conditions  and  Test  Procedures  For  Airborne 
Electronic/Electrical  Equipment  and  Instruments.  DO-1 60  or  its  predecessors 
DO-138  and  DO-1 04,  are  referenced  in  FAA  documents  when  it  is  necessary 
to  provide  detailed  instructions  on  how  a given  test  is  to  be  coordinated. 

No  item  is  subjected  to  all  tests  listed  in  D0-160.  During  the 
preparation  of  D0-160,  careful  attention  was  paid  to  permit  matching  of 
the  severity  of  the  tests  with  the  intended  use  of  the  items.  In  DO-138, 
aircraft  have  been  divided  into  three  classes:  Class  X,  aircraft  with  a 
maximum  ceiling  of  20,000  feet;  Class  which  includes  most  passenger  and 
cargo  aircraft  having  ceilings  of  50,000  feet;  and  Class  Z,  which  includes 
all  supersonic  aircraft.  Thirteen  categories  have  been  established  for 
equipment  based  on  variations  of  the  pressure,  temperature  that  the  equip- 
ment is  expected  to  encounter  in  service.  The  proximity  of  the  equipment 
to  the  engine,  and  the  use  of  pressurized  and  non-pressurized  compartments 
and  the  temperature  controlled  avionics  compartments  add  to  the  variations. 
MI L -STD-81 0C  is  not  as  easily  tailored  to  such  variations.  Many  TSO’s 
reference  DO-138,  which  is  not  as  inclusive  as  the  newer  D0-1 60. 
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A key  feature  of  MIL-STD-810C  ani  DO-7 60  is  that  they  are  both  "How 
to  test"  guides  and  dc  not  give  pass-or-fail  criteria.  Section  1 of  D0- 
160  states:  "The  selection  of  the  appropriate  environmental  conditions 
and  test  procedures  ->s  the  responsibility  of  those  who  write  the  per- 
formance standards  for  the  individual  airborne  electronic/electrical 
equipment  and  instruments."  Section  1 also  contains  the  following  Note: 
"In  earh  of  the  tost  procedures  contained  herein,  the  following  phase  will 
be  encountered  one  or  more  times:  'DETERMINE  COMPLIANCE  WITH  APPLICABLE 

EQUIPMENT  PERFORMANCE  STANDARDS'." 

Obviously,  when  two  manufacturers  state  that  they  have  built  a given 
type  of  avionics  equipment  that  has  "passed"  the  high  temperature  test  of 
D0--160,  a comparison  between  the  instruments  cannot  be  made  unless  the 
pass-fail  criteria  established  by  each  manufacturer  for  the  high  tempera- 
ture is  known.  One  manufacturer  might  accept  a much  greater  reduction  in 
accuracy  at  high  temperature  and  still  "pass"  the  test. 

Returning  to  the  example  used  in  Section  II,  comparing  TSO-ClOb 
requirements  for  an  altimeter  with  those  of  MIL-A-83212  for  the  AAU-27/A 
altimeter,  we  can  compare  the  required  environmental  testing.  First,  the 
TSO  altimeter  has  to  operate  through  a temperature  range  of  -30°C  to  50°C, 
while  the  AAU-27  must  operate  through  a range  of  -54°C  to  +71 °C.  A sig- 
nificant difference  here  is  that  the  AAU-27  is  tested  at  both  high  and 
low  temperatures,  and  while  the  TSO  altimeter  is  expected  to  operate  at 
temperatures  above  room  temperature,  it  is  not  tested  to  prove  this  cap- 
ability. Other  tests  which  the  AAU-27  must  pass,  which  the  FAA  does  not 
require  for  the  TSO  instrument  are:  salt  spray,  fungus,  sand  and  dust, 
shock  acceleration  and  radio  noise. 
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SECTION  IV 
Reliability  Testing 

Though  over  ninety  types  of  equipment  fall  into  categories  covered  by 
TSO's,  only  the  TSO  for  the  Ground  Proximity  Warning  System,  published  in 
1975,  has  a reliability  requirement  expressed  in  mean  time  between  failures 
(8000  hours).  For  all  other  TSO's  no  reliability  number  is  established. 

The  lack  of  formal  reliability  testing  does  not  mean  that  the  FAA  is  not 
concerned  with  the  reliability  of  commercial  equipment,  nor  does  it  mean 
that  commercial  equipment  is  inherently  unreliable.  On  the  contrary, 
commercial  equipment  is  considered  by  many  observers  to  be  more  reliable 
than  military  equipment.  The  environmental  tests  required  by  the  TSO's 
provide  a degree  of  reliability  testing.  Many  field  failures  are  a result 
of  environmental  stresses.  The  required  environmental  testing  can  expose 
many  of  these  design  weaknesses  and  result  in  redesign.  In  addition,  if 
a large  number  of  failures  occur  during  flight  operations,  the  FAA  may 
decide  to  review  the  quality  control  procedures  used  by  the  manufacturer. 
The  primary  factor  that  results  in  high  reliability  is  the  competitive 
environment  under  which  this  aviation  industry  operates.  A manufacturer 
who  consistently  produces  an  item  with  high  failures  rates  and  high 
repair  costs  will  not  survive  in  the  open  market.  Unfortunately,  the 
economics  of  the  market  place  takes  time  to  take  effect,  and  a marginal 
manufacturer  can  place  unreliable  equipment  on  the  market. 

The  commercial  avionics  industry  does  not  have  an  equivalent  relia- 
bility document  to  MIL-STD-781B  Reliability  Tests:  Exponential  Distribu- 
tion. The  FAA'c  recent  experience  with  the  reliability  requirements  im- 
posed on  the  ground  proximity  warning  system  parallels  the  DOD's  experience 
on  many  Hers:  reliability  demonstrated  in  the  test  lab  is  not  always 
duplicated  during  flight  operations. 
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SECTION  V 


Field  Reliability  Experience 

While  the  FAA  does  not  require  laboratory  reliability  demonstrations, 
the  FAA,  commercial  airlines  and  general  aviation  are  all  interested  in 
highly  reliable  avionic  equipment  from  both  a cost  and  safety  viewpoint. 
For  the  commercial  airlines,  the  primary  method  used  to  acquire  reliable 
equipment  is  to  make  the  manufacturer  contractually  responsible  for  the 
in-service  reliability  of  the  equipment.  This  is  done  through  a variety 
of  full  and  partial  guarantees  or  warranties.  One  increasingly  common 
practice  is  the  use  of  reliability  improvement  warranties  (RIW) . All  of 
these  practices  have  a common  trait,  financial  punishment  of  a contractor 
whose  equipment  is  unreliable. 

Records  of  reliability  performance  are  not  required  by  the  FAA,  but 
each  major  airline  develops  historical  data  from  avionics  equipment. 

These  records  are  closely  monitored  by  the  manufacturers.  While  there  is 
no  official  consolidated  airline  reliability  listing,  MTBF  performance 
values  and  modification  suggestions  are  commonly  shared  by  the  airlines. 
There  is  no  such  data  base  for  genera!  aviation  avionics. 

Detailed  analysis  of  reliability  differences  between  military  and 
commercial  avionics  may  be  found  in  several  reports.  One  enlightening 
and  current  report  is  the  Report  of  the  USAF  Scientific  Advisory  Board 
Guidance  and  Control  Panel  "Review  of  Aircraft  Inertial  Systems"  dated 
March  1975.  Another  useful  report  is  The  User-Technologist-Industrial 
Approach  to  Electronic  Equi pment  Specifications  and  Procurement  (El ec- 
tronic  X)  prepared  by  ARINC  Research  Corporation  for  the  Institute  of 
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Defense  Analyses  in  July  1973.  Both  reports  point  out  the  fact  that 
operational  MTBF  as  reported  by  the  military  should  riot  be  directly 
compared  with  commercial  MTBF.  The  Scientific  Advisory  Board's  report 
states  "...there  are  many  factors  in  low  USAF  reliability  — some  under 
AF  control,  others  due  to  fundamental  differences  in  operations." 

Commercial  MTBF  values  are  usually  based  on  verified  failures.  Pre- 
ventative maintenance  actions,  removals  to  gain  access  to  other  equipment 
and  reported  failures  that  are  not  later  verified,  are  not  counted  when 
calculating  the  MTBF.  The  Air  Force  MTBF  values  are  normally  obtained 
as  outputs  from  reports  required  by  AFM  66-1.  AFM  66-1  is  the  basis  of 
the  Air  Force  Logistic  Management  System,  and  is  not  intended  as  a method 
t"  develop  accurate  MTBF  value.  As  a result,  all  maintenance  actions 
normally  are  counted  as  failures  of  the  equipment.  Hence  removals  to 
facilitate  other  maintenance,  unverified  failures  and  preventative  main- 
tenance all  help  lower  the  calculated  MTBF.  In  some  cases  failures  have 
been  charged  against  a single  item  of  equipment  at  the  flight  line,  field 
shop  and  depot  for  a single  failure. 

For  inertial  navigation  systems,  the  FAA  requires  that  95%  of  the 
flights  have  errors  of  less  than  2 nautical  miles  along  track,  and  2 nm 
cross  track  per  hour.  This  is  equivalent  to  a circular  error  probable 
CEP  of  2.8  nm  per  hour.  For  the  F-15,  the  USAF  specified  a 3 nm/hr  system. 
McDonald  Douglas  required  a 2 nm/hr  system  from  the  subvendor  who  actually 
delivered  a system  with  an  average  error  of  0.7  nm/hr.  USAF  pilots  have 
become  accustomed  with  the  excellent  performance  of  the  system  and  will 
normally  "write-up"  the  system  if  the  error  is  over  2 nm/hr  even  though 
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the  performance  is  well  within  the  initially  required  performance.  These 
"soft"  failures  decrease  the  reported  MTBF.  Other  features  tending  to 
reduce  the  military  MTBF  are  increased  avionic  bay  temperatures,  gun  fire 
vibration,  and  a higher  ratio  of  equipment  turn-ons  per  1000  flight  hours. 
Airline  avionics  equipment  may  be  operated  3000  to  6000  hours  per  year 
while  most  military  aircraft  fly  600  hours  or  less  per  year.  As  a result, 
ageing  of  equipment  during  periods  of  non-operation  is  a more  significant 
factor  for  military  equipment. 

As  a specific  example  of  differences  between  MTBF  in  military  and 
commercial  use,  the  Carrousel  IV  reliability  may  be  used.  This  inertial 
navigation  set  is  widely  used  by  commercial  airlines  and  has  use  in  several 
special  military  programs. 

During  the  July  to  Dec  1975  oeriod,  the  Carrousel  IV  accumulated 
2,173,122  hours  of  operation.  Verified  relevant  failures  were: 


Control  Display  Unit 

279 

Mode  Selector  Unit 

3 

Battery 

21 

Inertial  Measuring  Unit 

1100 

Based  on  these  values,  the  commercial  service  MTBF  was  1549  hours. 

The  Carrousel  IV  is  used  by  the  military  on  the  EC-135J,  E-4A,  E-3A, 
C-9B,  HH-53  and  installed  for  transoceanic  missions  on  a pallet  in  KC-135 
and  EC-135  aircraft.  The  KC-135  and  EC-135  Palletized  Inertial  Navigation 
System  (PINS)  has  accumulated  the  greatest  number  of  operational  hours. 

The  PINS  accumulated  total  hours  from  4 March  74  to  29  February  76  were 
10,200.  There  were  21  verified  relevant  failures,  with  a resulting  MTBF 
of  486  hours.  If  AFM  66-1  data  had  been  used,  the  additional  unverified 
failures  would  have  reduced  the  MTBF  by  a factor  of  two.  MTBF  of  the 
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Carrousel  IV  in  other  military  installations  varies  from  a low  of  about 
150  hours  to  the  486  hours  reported  on  the  PINS  program. 

The  ARINC  Electronics  X report  contains  similar  rates  for  HF,  VHF 
and  ILS  equipment.  The  reliability  of  a particular  item  of  equipment 
during  commercial  service  is  two  to  three  times  the  reliability  of  the 
same  item  in  military  use. 
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SECTION  VI 

Impact  of  these  Factors  on  the  Program  Manager's  Decision 

There  have  been  several  recent  cases  where  the  military  services  have 
found  it  desirable  to  procure  fully  equipped  commercial  aircraft.  There 
have  been  numerous  cases  where  commercial  avionics  equipment  has  been 
selected  for  installation  in  a military  peculiar  aircraft  at  the  time 
of  initial  production  or  during  modification.  Instructions  in  the 
Request  for  Proposal  (RFP) , for  some  of  these  cases,  have  required  that 
the  avionics  and  instrument  equipment  be  qualified  in  accordance  with  the 
applicable  TSO. 

A review  of  the  method  used  by  the  FAA  to  grant  TSO  approval  of  the 
performance  requirements  of  a given  item  must  lead  the  program  manager 
to  the  conclusion  that  a simple  requirement  that  a TSO  approved  item  be 
furnished  is  woefully  inadequate.  Equipment  conforming  to  this  require- 
ment could  have  a performance  capability  far  exceeding  that  desired  or 
the  performance  could  be  below  that  required  for  the  intended  use.  For 
example,  an  attitude  indicator  meeting  the  requirements  of  TS0-C4c  could 
be  a vacuum  driven  indicator  with  a useable  pitch  range  of  plus  or  minus  30 
degrees  of  the  type  frequently  used  in  small  private  aircraft,  or  it  could  bt 
a DC-10  attitude  indicator  incorporating  ILS  data,  angle  of  attack  informa- 
tion and  a radar  altitude  presentation.  Both  instruments  could  meet  TS0-C4c, 
yet  the  difference  in  price  would  be  approximately  100  to  1.  The  proper 
selection  would  depend  upon  the  intended  use.  Certainly  there  is  no  objection 
to  the  use  of  a TSO  item,  but  a selection  should  be  made  after  a review  of  the 
particular  item  specification. 
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In  those  cases  where  the  only  specifications  available  are  those 
furnished  by  the  item  manufacturers,  the  selection  of  an  appropriate 
specification  for  inclusion  in  the  Request  for  Proposal  (RFR)  or  in 
the  contract  is  difficult.  There  may  be  several  acceptable  items  avail 
able  on  the  commercial  market,  but  no  single  specification  that  may  be 
referenced  that  would  permit  the  procurement  of  all  the  acceptable 
items.  As  a result,  the  program  manager  may  find  the  selection  of  the 
specification  unintentionally  places  the  procurement  in  a sole  source 
situation. 

The  selection  of  a commercial  item  for  a particular  military  use 
should  be  made  only  after  a careful  comparison  of  the  performance 
requirements  of  the  item  with  total  system  needs.  Just  because  a par- 
ticular performance  parameter  listed  in  the  specification  does  not  meet 
the  mission  requirements,  does  not  mean  that  the  item  will  not  function 
satisfactorily.  The  capability  of  the  equipment  may  exceed  the  spec- 
ification requirement.  Testing  may  show  the  equipment  capability 
exceeds  the  specification.  Most  manufacturers  would  be  willing  to  con- 
duct a few  tests  to  demonstrate  additional  capability  if  such  tests 
increase  the  possibility  of  sale. 

In  a similar  manner,  the  environmental  testing  required  by  the  TSO 
should  be  compared  with  those  tests  felt  necessary  for  the  specific 
application  being  considered.  In  those  cases  where  the  TSO  required 
environmental  tests  are  not  sufficiently  comprehensive,  additional 
test  results  may  be  available  for  the  manufacturer.  If  additional 
environmental  testing  is  still  considered  necessary,  program  funds 
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could  be  used  to  pay  for  additional  testing  or  the  tests  could  be 
conducted  "in-house"  at  a government  facility. 

Just  because  equipment  has  been  subjected  to  and  passed  all  the 
desired  environmental  tests  does  not  automatically  qualify  an  item 
from  an  environmental  standpoint.  An  essential  second  step  is  the 
evaluation  of  the  pass-fail  criteria  for  the  particular  test.  The 
degree  of  performance  deterioration  caused  by  an  adverse  environment 
may  be  considered  acceptable  by  the  manufacturer,  but  might  not  be 
acceptable  for  the  intended  application.  The  most  common  area  where 
commercially  available  equipment  will  fail  to  meet  military  environ- 
mental standards  is  in  the  electromagnetic  interference  (EMI)  area. 

After  all  available  test  information  is  reviewed,  it  may  occur 
that  the  commercial  equipment  still  faiis  to  meet  the  performance  or 
environmental  standards  in  some  regions.  At  this  point,  the  program 
manager  must  decide  to: 

1.  Permit  deviations  to  the  original  requirements; 

2.  Develop  an  item  of  equipment  specifically  for  his  program; 

3.  Utilize  available  military  inventory  equipment  if  adequate 
equipment  is  available; 

4.  Modify  the  existing  commercial  equipment  so  that  it  meets 
the  program  requirements. 

If  the  choice  is  modify  the  existing  commercial  equipment,  the 
initial  intent  of  buying  commercial  "off-the-shelf"  equipment  is  quickly 
lost.  Any  significant  modification  will  cause  the  equipment  to  lose 
TSO  or  type  certificate  approval  and  a new  series  of  performance  and 
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environmental  tests  must  be  performed.  Depending  on  the  type  of  modifi- 
cation, the  equipment  may  no  longer  be  suitable  for  normal  commercial  use 
and  a military  peculiar  item  results. 

A program  manager  must  make  decisions  on  acceptable  levels  of  reli- 
ability of  commercial  equipment  frequently  with  a minimum  of  information. 
The  situation  may  occur  at  the  time  of  original  full  scale  development  or 
when  a vendor  proposes  commercially  available  equipment  as  a replacement 
for  avionics  equipment  already  installed  that  has  high  support  costs 
because  of  low  reliability.  The  vendor  in  the  last  case  may  quote  a 
high  reliability  number  which,  if  achieved  in  military  service,  would  make 
the  replacement  of  the  existing  low  reliability  equipment  with  commercial 
equipment  cost  effective.  The  decision  before  the  program  manager  is  a 
difficult  one.  Such  decisions  are  normally  based  on  the  expected  long 
term  cost  to  the  government.  Most  of  the  costs  can  be  predicted  with  a 
reasonable  degree  of  accuracy  with  the  exception  of  the  repair  costs  which 
are  inversely  proportional  to  the  in-service  MTBF  of  the  equipment. 

As  stated  in  Section  IV,  reliability  tests  to  demonstrate  a laboratory 
MTBF  for  avionics  equipment  are  not  required  by  the  FAA  with  the  sole  ex- 
ception of  the  Ground  Proximity  Warning  System.  In  addition,  the  FAA 
does  not  track  the  reliability  of  avionics  equipment  presently  in  service. 
The  commercial  airlines  and  the  equipment  manufacturers  do  generate  MTBF 
figures  for  major  items  of  equipment  in  airline  service.  Only  very  im- 
precise information  is  available  on  avionics  equipment  installed  in  the 
light  aircraft  general  aviation  fleet. 
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Judgement  is  required  by  the  Program  Managers  and  his  supporting  engi- 
neering staff  in  the  use  of  the  available  MTBF  figures  because  of  the 
difference  in  preventative  maintenance  concepts,  reporting  procedures, 
failure  verification  methods  and  repair  concepts  discussed  in  Section  VI. 

In  addition  the  effect  the  environmental  differences  of  the  commercial 
and  military  installations  must  be  considered. 

While  in  some  cases  it  is  possible  to  forecast  that  the  commercial 
equipment  will  exhibit  a much  higher  MTBF  than  the  military  equipment, 
in  general,  however,  the  manufacturers  who  produce  military  equipment  are 
the  same  manufacturers  who  produce  for  the  commercial  aviation  market. 

For  similar  equipment  with  the  same  level  of  complexity  it  is  unlikely 
that  a dramatic  difference  in  inherent  reliability  exists.  Most  of  the 
differance  in  reliability  would  seem  to  be  accounted  for  by  the  environ- 
ment, reporting  procedures,  and  maintenance  factors  mentioned  earlier. 

The  example  of  differences  in  MTBF  of  the  Carrousel  IV  inertial  navigation 
set  in  commercial  service  vs  military  use  (1459  vs  486  hrs)  shows  that 
military  service  MTBF  does  not  always  parallel  commercial  experience. 
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SECTION  VII 
Conclusions 

The  same  degree  of  care  must  be  taken  when  selecting  commercial 
avionics  equipment  for  a military  use  as  must  be  taken  when  selecting 
an  item  of  qualified  government  furnished  equipment  for  a new  applica- 
tion. 

The  key  points  influencing  the  selection  and  use  of  "off-the-shelf" 
commercial  equipment  are: 

1.  The  FAA  approval  documents  (TSO's)  are  not  intended  for 
nor  are  they  suitable  for  use  as  procurement  specifications. 

2.  To  obtain  FAA.  approval,  an  item  of  avionics  equipment  need 
meet  only  minimum  performance  standards,  hence  the  item  may  not  have  the 
necessary  capability  to  permit  utilization  of  all  the  intended  perfor- 
mance of  the  weapon  system. 

3.  Airframe  manufacturers  and  commercial  airlines  purchase 
avionics  equipment  to  airframe  company  specification,  manufacturers 
specification  or  association  specification  (e.g.,  ARINC,  RTCA).  These 
specifications  may  be  suitable  for  military  procurements,  but  a careful 
review  must  be  made  to  ensure  that  the  performance  specified  is  adequate 
for  the  intended  military  use. 

4.  The  fact  the  performance  specification  doesn't  detail  some 
necessary  performance  characteristics  or  the  performance  values  listed 
are  too  restrictive  doesn't  mean  the  equipment  must  be  rejected.  The 
equipment  may  have  capabilities  exceeding  the  specification. 
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5.  The  program  manager  should  consider  funding  performance  test- 
ing to  determine  if  the  equipment  has  capabilities  exceeding  the  perfor- 
mance specification  if  the  additional  capability  is  required  for  the 
intended  use. 

6.  RTCA  Document  D0-160,  Environmental  Conditions  and  Test 
Procedures  for  Airborne  Eiectronic/Electrical  Equipment  and  Instruments 
specifies  a broad  range  of  environmental  test  conditions  and  is  generally 
equivalent  to  MIL-STD-810C,  Environmental  Test  Methods.  Both  documents 
are  "How-to-Test"  guides  and  do  not  contain  accept  or  reject  criteria. 

7.  The  equipment  specifications  must  be  reviewed  to  determine 
if  the  required  environmental  tests  are  adequate  to  assure  adequate  cap- 
ability in  the  expected  operational  environment.  In  addition,  the  per- 
formance tolerances  of  the  instrument  during  environmental  tests  must  be 
reviewed  to  determine  the  extent  of  performance  degradation  and  to  deter- 
mine if  this  degraded  operation  is  acceptable  for  the  intended  use. 

8.  The  program  manager  should  consider  funding  additional  envi- 
ronmental tests  if  necessary  to  establish  the  suitability  of  the  equipment 
for  the  military  environment  in  which  it  is  to  operate. 

9.  The  FAA  does  not  require  a laboratory  reliability  test 
demonstration,  nor  does  the  FAA  track  service  reliability  of  avionics 
equipment. 

10.  The  MTBF  of  equipment  in  commercial  airline  service,  as 
reported  by  manufacturers  and  industry  associations,  has  been  collected 
using  failure  criteria  different  than  the  military  services.  In  addition 
the  operational  environment,  preventative  maintenance  concepts  and 
flight  line  and  depot  repair  procedures  effect  the  reliability  of  the 
equipment. 
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11.  Generally,  commercial  equipment  used  in-military  aircraft 
exhibits  a significantly  lower  MTBF  in  military  service  when  compared 
to  the  MTBF  in  commercial  airline  service. 
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